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ABSTRACT 

The  U.S.  Air  Force  aircraft  maintenance  depots  face  complex  operating  environments  due  to  the  diversity 
of  aircraft  or  mission  design  series  (MDS)  maintained  by  each  depot  and  the  variability  of  maintenance 
requirements  for  each  MDS.  Further  complicating  their  operations  is  the  variability  of  maintenance  ac¬ 
tions  required  from  one  aircraft  to  another  within  each  MDS  and  a  highly  specialized  workforce  that  has 
inherent  inflexibility  to  compensate  for  the  workload  variability.  Air  Force  Materiel  Command  is  review¬ 
ing  maintenance  personnel  multi-skilling  as  a  method  to  efficiently  absorb  the  variability  of  workload  and 
maintenance  requirements  between  aircraft.  Using  a  simulation  built  in  ARENA  14®,  we  studied  the  F- 
22  Heavy  Maintenance  Modification  Program  through  a  series  of  designed  experiments.  Our  study  ana¬ 
lyzes  whether  using  a  multi-skilled  workforce  impacts  the  productivity  of  depot  maintenance  personnel 
through  simulation  of  several  multi-skilling  policies. 

1  INTRODUCTION 

The  Air  Force  faces  significant  fiscal  challenges  in  the  coming  years  with  dwindling  defense  budgets  and 
increasing  procurement  and  sustainability  costs  associated  with  more  technologically  advanced  weapons 
systems  and  an  aging  legacy  fleet.  The  Air  Force  Sustainment  Center  and  the  three  Air  Logistics  Com¬ 
plexes  ( ALCs)  under  their  purview  are  at  the  forefront  of  the  battle  to  affect  the  cost  curve  for  sustainment 
operations.  The  aircraft  maintenance  depot  activities  at  Ogden  ALC,  Oklahoma  City  ALC  and  Wamer- 
Robins  ALC  face  complex  operating  environments  due  to  the  diversity  of  aircraft  or  mission  design  series 
(MDS)  maintained  by  each  depot  and  the  variability  of  maintenance  requirements  for  each  MDS.  Further 
complicating  their  operations  is  the  variability  of  maintenance  actions  required  from  one  aircraft  to  anoth¬ 
er  within  each  MDS  and  a  highly  specialized  workforce  that  has  inherent  inflexibility  to  compensate  for 
the  workload  variability. 

Multi-skilling  is  one  proposal  that  Air  Force  Materiel  Command  (AFMC)  is  reviewing  to  absorb  the 
variability  of  workload  and  maintenance  requirements  between  aircraft.  Multi-skilling  is  defined  as  “a 
position  that  combines  two  or  more  journeyman,  full  performance  or  higher  level  skills  in  the  same  pay 
plan  in  which  formal  on-the-job  or  classroom  training  is  required”  (Federal  Service  Impasse  Panel  1997). 
In  1 993  the  Oklahoma  City  ALC  implemented  a  multi-skill  program,  but  the  program  never  materialized 
in  the  way  designed  and  therefore  had  negligible  results.  The  question  remained  unanswered  on  whether 
Air  Force  depot  operations  can  more  cost  effectively  use  their  work  force  through  multi-skilling. 
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For  that  reason,  AFMC  requested  an  analysis  of  the  F-22  Heavy  Maintenance  and  Modification  Program 
to  estimate  the  potential  benefits  of  a  multi-skilled  workforce. 

We  begin  by  introducing  the  problems  attributed  to  a  task  centric  maintenance  workforce  within  the 
F-22  Heavy  Maintenance  Modification  Program.  We  then  discuss  modeling  the  F-22  depot  maintenance 
process  and  our  verification  and  validation  process.  Finally  we  present  the  results  of  our  study  and  pro¬ 
vide  recommendation  for  future  research. 

2  PROBLEM  DEFINITION 

The  F-22  Heavy  Maintenance  Modification  Program  consists  of  several  Federal  Wage  Series  occupations 
that  perform  maintenance  on  the  F-22  depot  production  line.  However,  the  F-22  depot  is  currently  expe¬ 
riencing  indirect  labor  hours  and  overtime  rates  above  their  target  levels  in  their  “current  state”  operations 
today.  They  are  furthermore  concerned  that  their  overtime  costs  may  increase  in  a  planned  “future  state” 
when  required  annual  aircraft  throughput  will  increase.  A  business  case  analysis  completed  by  Ogden 
ALC  in  2012  hypothesizes  that  the  problems  lies  in  the  Painter  (Low  Observable  or  LO)  workforce  con¬ 
straints  and  associated  downtime  of  other  occupations  awaiting  LO  task  completion. 

The  cost  associated  with  overtime  and  idle  personnel  created  by  the  painter  labor-hour  constraint 
appears  to  create  an  opportunity  for  potential  productivity  gains  by  multi-skilling  other  labor  specialties  to 
increase  painter  labor  hours  availability.  No  substantive  quantitative  analysis  exists  on  which  mainte¬ 
nance  specialties  within  F-22  depot  operations  are  favorable  for  multi-skilling  into  the  painter  specialty. 
Additionally,  no  quantitative  analysis  exists  to  estimate  the  aircraft  throughput  and  employee  utilization 
impacts  of  multi-skilling  in  depot  operations  other  than  a  limited  simulation  of  the  KC-135  depot  process 
by  Levien  (2010). 

3  LITERATURE  REVIEW 

3.1  Simulation 

The  complexity  of  the  F-22  Heavy  Maintenance  Modification  Program  and  the  AFMC  desire  to  review 
multiple  multi-skilling  policies  led  us  to  select  simulation  to  model  the  system.  Banks  et  al.  (2010)  ex¬ 
plain  that  many  real-world  systems  are  so  complex  that  models  of  the  systems  are  virtually  impossible  to 
solve  mathematically  and  simulation  allows  for  the  study  of  those  systems.  Law  (2007)  notes  that  it  is 
rarely  necessary  to  include  every  element  of  the  system  in  the  model  in  order  to  make  effective  decisions, 
and  in  some  cases  can  cause  excessive  model  run  time,  causing  missed  deadlines  and  obscuring  important 
factors.  Including  the  relevant  details  in  the  model  facilitates  the  ability  of  the  model  to  provided  system 
insights  that  are  useful  for  decision  making. 

The  reasons  for  using  simulation  can  be  negated  if  the  model  does  not  reflect  real  system  behaviors  or 
if  the  outputs  do  not  provide  useful  insights  into  system  behaviors.  Deliberate  verification  and  validation 
(V  &  V)  ensures  the  model  reasonably  depicts  the  real  system  and  its  associated  behaviors.  Carson 
(2005),  Law  (2007),  Banks  et  al.  (2010),  and  Little  (2004)  emphasize  the  importance  of  involving  manag¬ 
ers  and  their  teams  in  validating  the  model.  We  did  this  with  two  trips  to  the  depot  and  with  periodic  tel¬ 
ephone  calls  and  email  exchanges  with  the  depot  team. 

3.2  Depot  Maintenance  and  Multi-skilling  Simulation 

Levien  (2010)  simulated  the  KC-135  depot  maintenance  process,  focusing  specifically  on  whether  multi¬ 
skilling  improved  a  designated  major  job  within  the  process.  He  considered  only  three  of  the  six  mainte¬ 
nance  specialties  used  in  the  major  job  tasks.  Model  validation  proved  quite  difficult  for  him — possibly 
because  he  was  only  able  to  model  a  portion  of  the  aircraft  repair  network.  He  was  still  able  to  provide 
interesting  insights  on  multi-skilling  personnel  that  helped  us  to  focus  our  work. 

Park  (1991)  examined  cross-training  in  a  dual  resource  constrained  shop.  He  studied  dual  resource 
constrained  systems,  where  both  personnel  and  resources  were  needed  to  complete  a  job  but  excess  ma¬ 
chines  exist.  He  used  a  SLAM  simulation  to  model  five  work  centers  at  varying  levels  of  cross-trained 
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personnel  and  with  differing  work  dispatch  policies,  and  concluded  that  “The  minimum  introduction  of 
worker  cross-training  showed  the  most  significant  improvement,  and  subsequent  increase  in  cross-training 
had  diminishing  return”.  His  results  support  the  business  case  analysis  completed  by  the  Ogden  ALC  in 
2012. 


3.3  Marginal  Analysis 

The  objective  of  our  research  was  to  investigate  whether  multi-skilling  personnel  policies  could  provide 
better  flowtime  and  cost  outcomes  than  the  status  quo  overtime  policy.  Marginal  analysis  is  a  form  of  op¬ 
timization  that  assists  in  selecting  from  among  differing  technically  efficient  alternatives  (de  Neufville, 
1990).  Marginal  analysis  is  useful  for  investigating  the  system  design  through  simulation  if  the  costs  of 
inputs  are  readily  available  and  allows  for  selection  of  the  alternative  that  gives  the  best  return  on  inputs 
invested.  Ysebaert  (2011)  used  marginal  analysis  by  implementing  a  “shopping  list”  approach  to  gauge 
the  improvements  of  Adjusted  Stock  Level  panels  for  the  F-22.  Her  marginal  analysis  measured  aircraft 
availability  improvement  versus  the  cost  of  an  additional  panel.  Ysebaert’s  results  provided  valuable  in¬ 
sights  into  the  aircraft  availability  improvements  possible  with  purchases  of  additional  panels  as  well  as 
the  cost  of  those  improvements.  Similar  methods  are  used  in  the  Aircraft  Sustainability  Model  by  com¬ 
paring  optimal  spares  mixes  based  on  a  target  metric  defined  by  the  user  (Slay  et  al.  1996).  Both 
Ysebaert  (2011)  and  Slay  et  al.  (1996)  use  marginal  analysis  based  on  simulation  outputs  to  decide  the 
“benefit  per  dollar”,  where  the  scenarios  with  the  highest  benefit  shoidd  be  at  the  top  of  the  “shopping 
list”. 

For  our  system,  the  goal  is  not  just  to  compare  alternatives  but  also  to  select  the  best  of  the  differing 
alternatives  through  statistically  proven  ranking-and-selection  (R  &  S)  methods.  Kim  and  Nelson  (2006) 
give  four  comparison  problems  that  arise  in  simulation  studies  including  comparing  alternatives  against  a 
standard,  selecting  the  best  performing  system,  selecting  the  system  with  the  highest  probability  of 
performing  the  best,  and  selecting  the  largest  probability  of  success.  Their  emphasis  is  on  the  constraint 
applied  to  probability  of  the  correct  selection  (Kim  and  Nelson,  2006).  Nelson  et  al.  (2001)  offer 
procedures  for  selecting  the  best  system  and  experimental  results  that  validate  its  ability  to  find  the  best 
system  with  a  confidence  level  of  at  least  1-a.  Banks  et  al.  (2010)  provide  an  automated  tool  that 
implements  these  procedures  in  the  SimulationTools.xls  spreadsheet  at  www.bcnn.net. 

4  METHODOLOGY 

4.1  Goal  and  System  of  Interest 

Our  goal  was  to  quantitatively  examine  multi-skilling  and  gain  insight  into  specialty  selection  for  multi¬ 
skilling.  We  built  a  simulation  in  ARENA  14®  with  outputs  capturing  aircraft  throughput,  average  flow 
days,  employee  utilization  by  specialty,  and  dock  usage.  Prior  to  delving  into  the  details  of  the  model,  we 
briefly  discuss  the  system  of  interest  and  the  model  scope. 

The  entity  of  primary  interest  is  individual  aircraft  that  enter  and  exit  the  system.  Aircraft  must  be  pe¬ 
riodically  inspected  and  overhauled  at  a  depot.  The  system  boundaries  for  our  research  are  defined  as 
starting  when  depot  maintenance  personnel  begin  scheduled  maintenance  actions  on  an  aircraft  and  as 
stopping  when  all  maintenance  actions  are  complete.  Each  aircraft  arrives  at  scheduled  intervals  deter¬ 
mined  by  engineering  specifications  and  based  on  collaborative  planning  between  the  aircraft  depot  plan¬ 
ners  and  organizations  that  own  the  aircraft.  Each  aircraft  arrives  with  a  preplanned  number  of  mainte¬ 
nance  actions  scheduled  based  on  modifications  required,  the  delayed  discrepancies  identified  for 
completion  and  any  other  maintenance  requirements  found  during  the  depot  process.  The  aircraft  is 
scheduled  to  be  in  the  system  for  a  predetermined  number  of  workdays  based  on  maintenance  man-hour 
requirements  with  all  maintenance  actions  scheduled  for  completion  within  this  time  frame.  The  planned 
flow  days  for  each  aircraft  are  broken  down  into  seven  measureable  segments  referred  to  as  gates.  The 
seven  gates  exist  as  milestones  within  the  process  to  focus  maintenance  efforts  on  meeting  predetermined 
timelines  and  goals  for  process  completion.  Management  and  production  supervisors  restrict  personnel  to 
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working  only  maintenance  actions  required  within  the  gate  the  aircraft  is  currently  assigned  and  mainte¬ 
nance  actions  within  the  next  gate  do  not  begin  until  all  actions  from  the  previous  gate  are  completed, 
with  rare  exceptions.  There  are  six  Federal  Wage  Series  (FWS)  occupational  specialties  that  complete  a 
majority  of  the  maintenance  activities  within  each  of  the  seven  gates.  A  critical  path  analysis  by  the  depot 
planners  identified  maintenance  activities  of  these  six  specialties  as  critical  path  tasks  and  focused  our  re¬ 
search  efforts  on  the  six  identified  specialties. 

4.2  Model  Scope  and  Inputs 

We  initially  considered  modeling  only  a  few  of  the  seven  gates  to  manage  the  model  size  and  complexity. 
Modeling  two  or  three  gates  will  give  an  idea  of  output  behaviors  associated  with  those  gates  as  multi¬ 
skilling  is  implemented  and  may  give  an  accurate  representation  of  the  overall  system  behaviors  associat¬ 
ed  with  multi-skilling.  The  problem  with  this  approach  is  that  man-hour  requirements  per  specialty  vary 
greatly  between  each  gate  and  isolating  the  model  to  certain  gates  may  inaccurately  show  higher  man¬ 
power  availability  than  actually  exists  within  the  entire  system  if  the  wrong  gates  are  included.  Further 
complicating  the  problem  of  identifying  gates  for  inclusion  is  that  subject  matter  expert  inputs  identify  pe¬ 
riods  when  excess  resource  (personnel)  capacity  exists  due  to  inactivity  in  a  gate  (i.e.  no  aircraft  in  the 
gate).  Management  and  planners  seem  unable  to  forecast  the  gates  and  periods  when  this  excess  capacity 
will  arise,  leading  to  our  decision  to  model  all  seven  gates  versus  scoping  the  model  to  a  few.  After  de¬ 
ciding  to  model  the  entire  system,  our  next  step  was  to  decide  on  the  level  of  detail  to  include  within  the 
model. 

The  highest  level  of  detail  is  to  model  the  system  at  task  level  in  order  to  most  accurately  capture  the 
benefits  of  multi-skilling.  However,  our  time  constraints,  the  sheer  number  of  maintenance  tasks  (three  to 
four  thousand)  required  on  each  aircraft,  and  the  variability  of  the  number  of  tasks  from  one  aircraft  to  the 
next  make  modeling  all  tasks  impractical  for  this  study.  Since  variability  of  man-hour  requirements  is  the 
most  significant  challenge  that  multi-skilling  is  proposed  to  address,  the  key  factor  in  determining  the  lev¬ 
el  of  aggregation  for  the  model  is  the  ability  to  simulate  multiple  types  of  aircraft  from  a  man-hour  re¬ 
quirement  perspective.  With  this  in  mind,  we  identified  the  data  that  managers  and  planners  use  to  fore¬ 
cast  current  requirements.  F-22  depot  planners  had  already  worked  with  a  team  from  Clemson  University 
to  produce  an  automated  scheduling  tool  to  identify  man-hour  requirements  per  specialty  per  day.  The 
Clemson  tool  produces  the  aggregate  man-hour  requirements  per  specialty  per  flow  day  for  an  aircraft  as 
well  as  other  scheduling  outputs.  The  tool  pulls  data  from  the  depot’s  Programmed  Depot  Maintenance 
Scheduling  System,  including  the  standard  forecast  hours  based  on  the  projected  maintenance  require¬ 
ments  of  an  aircraft  or  the  actual  hours  used  to  complete  an  aircraft  previously  in  the  system.  Most  deci¬ 
sions  within  depot  operations  are  viewed  from  the  man-hour  requirement  perspective  and  so  we  designed 
our  simulation  model  to  incoiporate  that  paradigm  by  using  the  Clemson  tool  inputs.  Therefore,  the  model 
was  built  with  the  intent  of  simulating  the  flow  of  aircraft  through  the  system  with  maintenance  actions  on 
each  aircraft  being  performed  based  on  man-hour  requirements  for  each  of  the  six  specialties  per  day. 

4.3  Model  Description 

The  ARENA  14®  simulation  was  built  to  reflect  the  depot  system  of  interest  in  two  states,  the  current  and 
projected  future  state  of  the  system.  The  two  models  are  identical  with  only  minor  input  differences 
based  on  changes  in  projected  workloads  and  personnel  resources  available. 

The  aircraft  arrive  at  constant  intervals  to  reflect  the  flow  rate  required  to  reach  target  aircraft 
throughput  for  each  year.  Upon  arrival,  the  aircraft  flows  through  eight  main  sub-models  including  a 
Characteristics  sub-model  and  additional  sub-models  for  each  of  the  7  Gates  as  depicted  in  the  top  of  Fig¬ 
ure  1. 

The  Characteristics  sub-model  randomly  decides  which  of  four  possible  types  of  aircraft  are  arriving 
using  probabilities  based  on  depot  data.  Each  aircraft  type  represents  a  different  level  of  required  mainte¬ 
nance  activity  and  is  based  on  four  actual  aircraft  histories:  one  with  a  minor  amount  of  required  work; 
one  with  a  large  amount  of  required  work,  and  two  others  that  represented  intermediate  levels  of  required 
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work.  The  random  selection  of  the  four  types  of  aircraft  induces  man-hour  requirement  variability  into 
the  system  and  is  a  critical  factor  to  accurately  reflect  the  variability  of  real  system  inputs.  The  Character¬ 
istics  sub-model  also  “deconflicts”  arrivals  based  on  the  constraint  to  have  only  a  certain  number  of  air¬ 
craft  within  the  system  at  any  time.  After  exiting  the  Characteristics  sub-model,  the  aircraft  is  considered 
to  be  work  in  process  and  enters  Gate  1 . 

The  Gate  1  sub-model  is  broken  down  into  six  main  processes  and  two  main  lower-tier  sub-models 
with  several  record  functions  used  to  tally  statistics  throughout  the  gate.  The  aircraft  is  separated  into  six 
different  entities  as  shown  between  points  A  and  B  in  Figure  1.  The  entities  flow  into  separate  paths  rep¬ 
resenting  the  six  FWS  specialties  of  interest.  In  queuing  theory  terms,  each  path  corresponds  to  a  differ¬ 
ent  type  of  server  required  to  complete  a  specific  maintenance  activity.  Using  a  different  path  design  al¬ 
lows  resource  pooling  based  on  the  type  of  server.  When  multi-skilling  is  introduced  the  server  is  able  to 
complete  maintenance  hours  associated  with  multiple  resource  pools  versus  just  one.  The  paths  are  repli¬ 
cas  of  each  other  with  minor  equation  differences. 


Each  path  contains  a  Visual  Basic  block  that  uses  custom  code  to  read  in  the  specialty’s  daily  mainte¬ 
nance  hour  requirements  from  the  Clemson  scheduling  tool.  The  maintenance  hour  requirements  are  as¬ 
signed  based  on  the  type  of  plane  assigned  and  the  aircraft’s  current  flow  day.  Then  the  aircraft  flows 
through  a  sub-model  that  decides  the  number  of  personnel  to  seize  by  a  subsequent  process  module.  The 
process  module  uses  resource  sets  to  seize  personnel.  Each  resource  set  contains  the  primary  maintenance 
specialty  required  and  a  multi-skilled  specialty  for  each  of  the  other  five  specialties  paired  with  the  prima¬ 
ry  specialty.  The  personnel  resources  are  seized  until  the  hourly  requirements  are  met  for  the  day  based 
on  the  hourly  requirement  divided  by  the  number  of  personnel  seized. 

Once  maintenance  requirements  are  met,  the  second  half  of  the  gate  (Figure  1,  points  B  to  C)  re¬ 
strains  the  six  identical  entities  to  remain  within  a  certain  number  of  flow  days  of  each  other.  It  then  re¬ 
loops  the  entities  back  to  the  beginning  of  the  gate  if  the  entity  requires  more  flow  days  within  the  gate 
and  batches  the  entities  back  into  a  single  aircraft  before  allowing  it  to  exit  the  current  gate.  When  all  re- 
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quirements  are  met  for  each  of  the  seven  gates  the  flow  days  and  other  output  statistics  are  recorded  and 
the  aircraft  exits  the  system. 

The  depot  operates  today  on  a  three-shift  schedule.  We  set  the  personnel  capacity  for  each  specialty 
to  the  value  of  one  shift  of  personnel.  The  model’s  hours-per-day  value  was  adjusted  to  reflect  the  dura¬ 
tion  of  three  shifts  worth  of  labor  hours,  mimicking  the  number  of  hours  that  personnel  would  be  availa¬ 
ble  to  complete  maintenance  in  a  given  day.  This  method  reflects  the  same  personnel  and  labor  hours 
available  as  scheduling  three  equal  personnel,  equal  duration  shifts.  The  only  difference  is  the  shift 
change  is  removed  and  the  personnel  continue  on  the  job  instead  of  instantaneously  stopping  mainte¬ 
nance,  changing  personnel,  and  restarting  the  same  maintenance  (processing)  with  the  same  type  of  per¬ 
son  or  resource. 

4.4  Model  Verification  and  Validation 

Verification  and  validation  of  a  simulation  model  ensures  the  design  and  function  accurately  represents 
the  behaviors  of  the  real  system.  According  to  Carson  (2005),  “The  result  of  the  V  &  V  phase  is  a  veri¬ 
fied,  validated  model  that  is  judged  to  be  accurate  enough  for  experimentation  purposes  over  the  range  of 
system  designs  contemplated”.  For  this  reason,  we  built  the  simulation  model  in  a  piecewise  fashion  with 
verification  and  validation  methods  being  used  continually  throughout  the  process.  Final  validation  of  the 
model  included  comparison  with  historical  data  and  discussions  with  depot  subject  matter  experts.  The 
simulation  produced  outputs  within  2  percent  of  the  expected  values  from  historical  data,  which  we 
deemed  satisfactory. 

The  warm-up  period  is  100  days  for  the  current  state  in  order  to  load  work- in-process  (W1P)  aircraft 
into  the  model  and  to  allow  enough  time  for  the  first  aircraft  to  exit  the  system  at  the  start  of  statistics  col¬ 
lection.  The  future-state  warm-up  period  is  200  days.  Our  reasoning  for  allowing  the  first  aircraft  to  exit 
the  system  is  that  personnel  resource  constraints  do  not  affect  the  W1P  aircraft  until  a  truly  representative 
quantity  of  aircraft  are  in  the  model,  which  enables  the  first  aircraft  to  flow  through  the  system  at  an  unre¬ 
alistic  rate  with  seemingly  minimal  resource  constraints.  The  warm-up  period  is  added  to  the  replication 
run  length. 

We  used  the  flow  days  coefficient  of  variation  (CV)  to  determine  run  length  and  number  of  replica¬ 
tions,  by  varying  the  run  length  from  3  to  10  years  and  replications  from  5  to  30.  We  found  that  15  repli¬ 
cations  of  1 0-year  run  lengths  per  scenario  provided  the  best  tradeoff  of  acceptable  run  times  versus  out¬ 
put  variation. 

4.5  Experimentation 

We  used  marginal  analysis  together  with  the  Select-the-Best  Procedure  from  Banks  et  al.  (2010).  The  Se- 
lect-the-Best  Procedure  was  used  to  select  the  scenario  with  the  best  improvement  over  the  baseline  using 
a  95  percent  confidence  level  and  an  indifference  level  of  two  days.  Our  “current  state”  experiments  in¬ 
clude  a  personnel-add  experiment  and  several  multi-skilling  policy  experiments.  The  “future  state”  ex¬ 
periments  include  a  personnel  reduction  experiment  and  reduced  workforce  multi-skilling  experiments. 

The  first  current  state  experiment  mimics  adding  overtime  through  the  addition  of  personnel  and  the 
same  method  is  used  in  the  future  state  experiments  to  re-add  personnel.  We  add  one  person  by  specialty 
for  each  run,  reflecting  the  addition  of  three  additional  personnel  per  day  or  one  person  in  each  of  the 
three  balanced  shifts  (1800  labor  hours/person/year).  Each  experimental  level  contains  six  runs;  one  run 
(scenario)  for  each  specialty.  Note  that  only  one  specialty  has  added  personnel  from  the  baseline  for  each 
run.  The  specialty  that  shows  the  most  improvement  in  flow  days  from  the  additional  personnel  using  the 
Select-the-Best  Procedure  is  retained  as  the  new  baseline  for  the  next  comparison.  This  method  of  mar¬ 
ginal  analysis  is  similar  to  Ysebaerf  s  (2011)  “shopping  list”  method  but  uses  the  Select-the-Best  Proce¬ 
dure  to  choose  the  best  system.  The  current  state  experiment  shows  the  impact  of  adding  personnel  (over¬ 
time  hours)  on  flow  days  and  provides  a  comparison  for  multi-skilling  improvements. 

The  multi-skilling  experiments  consider  different  multi-skilling  policies  by  altering  the  maintenance 
specialty  processes  that  a  set  of  personnel  (a  particular  specialty)  within  the  system  can  complete.  Then 
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the  personnel  add  and  multi-skilling  experiment  results  are  compared  to  identify  the  policy  with  the  best 
system  performance.  The  personnel  add  and  multi-skilling  methods  are  similar  in  both  our  current  and 
future  state  experiments.  The  multi-skilling  experiments  consider  pairings  proposed  in  the  2012  Ogden 
ALC  business  case  analysis  and  pairings  selected  based  on  utilization  rates  and  resource  capacity. 

5  RESULTS  AND  ANALYSIS 

We  discuss  our  results  for  both  the  system  current  and  future  states.  Note  that  the  principal  difference  be¬ 
tween  the  current  state  and  the  planned  future  state  systems  are  increases  in  workload,  personnel  re¬ 
sources,  and  changes  in  the  percentages  for  the  types  of  aircraft  arriving. 

5.1  Current  State  Results 

The  current  state  experiments  reflect  the  depot’s  throughput  today  and  provide  a  comparison  between 
overtime  use  and  multi-skilling  different  number  of  personnel.  Table  1  shows  that  multi-skilling  a  work¬ 
force  using  targeted  (experiment  treatments  that  build  on  insights  from  prior  model  runs)  multi-skilling 
policies  with  no  efficiency  losses  enable  aircraft  throughput  similar  to  the  best  overtime  scenario.  Our 
cost  figures  use  2012  rates  for  overtime  or  an  additional  $2.03  hourly  wage  rate  for  multi-skilled  person¬ 
nel.  The  annual  cost  difference  between  the  best  targeted  multi-skilling  policy  (Level  Cl)  and  overtime 
policy  (Level  A8)  is  approximately  $1,152,000.  The  second  best  targeted  multi-skilling  policy  (Level 
C2)  provides  a  two-flow-day  difference  from  the  best  overtime  result  with  an  annual  cost  difference  of 
$1,349,000.  Note  that  Cl  and  C2  are  not  statistically  different  from  each  other.  In  the  two  best  targeted 
multi-skilling  experiment  levels,  the  flow  day  outputs  show  a  12  to  13  percent  decrease  in  flow  days  from 
the  baseline  model,  falling  2  to  3  percent  below  the  flow  day  target  (goal)  for  the  system. 


Table  1:  Current  state  results  summary. 


Current  State  Experiments  -  Results  Summary 

Experiment  Name  (Level) 

Flow  Days 

Delta 

w/Baseline 

Annual 

Throughput 

Overtime  / 
Multi-skill  Hrs 

Annual 

Cost 

Baseline 

133.99 

N/A 

9.90 

N/A 

N/A 

Personnel  Add  -  Overtime  (A8) 

115.5 

-18.5 

11.30 

48,600 

$1,965,870 

Targeted  Multi-skilling  (Cl) 

116.29 

-17.7 

11.27 

400,950 

$813,929 

Targeted  Multi-skilling  (C2) 

117.55 

-16.4 

11.14 

303,750 

$616,613 

Targeted  98%  Efficient  (Cl) 

119.8 

-14.2 

11.03 

400,950 

$813,929 

Targeted  98%  Efficient  (C2) 

120.92 

-13.1 

10.87 

303,750 

$616,613 

Personnel  Add  -  Overtime  (A6) 

120.96 

-13.0 

10.92 

32,400 

$1,310,580 

Targeted  Multi-skilling  95%  Efficient  (Cl) 

123.16 

-10.8 

10.77 

400,950 

$813,929 

Personnel  Add  -  Overtime  (A4) 

123.54 

-10.5 

10.67 

21,600 

$873,720 

Targeted  Multi-skilling  95%  Efficient  (C2) 

124.9 

-9.1 

10.60 

303,750 

$616,613 

Personnel  Add  -  Overtime  (A3) 

125.26 

-8.7 

10.58 

16,200 

$655,290 

Personnel  Add  -  Overtime  (Al) 

129.82 

-4.2 

10.24 

5,400 

$218,430 

Targeted  Multi-skilling  90%  Efficient  (Cl) 

130.22 

-3.8 

10.20 

400,950 

$813,929 

Targeted  Multi-skilling  90%  Efficient  (C2) 

131.25 

-2.7 

10.15 

303,750 

$616,613 

AP  All  Multi-skill  100%  (Bl) 

135.12 

1.1 

9.70 

400,950 

$813,929 

Average  Flow  Day  Target  w/OT 

120.64 

Expected  OT  Cost  for  Real  System 

$1,240,116 

As  workers  become  responsible  for  multiple  skills,  they  risk  a  decline  in  individual  skill  proficiency. 
However,  we  found  that  reductions  in  efficiency  levels  to  98  percent  (reflective  of  2  percent  longer  pro¬ 
cessing  times)  for  the  targeted  policies  still  provide  aircraft  throughput  timelines  that  fall  below  the  flow 
day  goal  with  overtime.  These  policies  indicate  a  cost  difference  between  multi-skilling  and  overtime  of 
$694,000  annually  for  the  targeted  multi-skilling  second-best  policy  and  $497,000  annually  for  the  best 
performing  policy,  both  at  98  percent  efficiency.  Further  reducing  the  workforce  efficiency  level  dimin¬ 
ishes  the  cost  differences  between  overtime  and  targeted  multi-skilling  with  break-even  points  occurring 
at  95  percent  efficiency  for  both  targeted  policies.  This  indicates  that  maintenance  actions  can  take  up  to 
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5  percent  longer  before  the  overtime  and  multi-skilling  policies  become  equally  preferable  in  terms  of 
cost.  One  point  of  interest  is  that  the  second  best  targeted  multi-skilling  experimental  level  includes  48 
fewer  multi-skilled  personnel  than  the  best  policy.  If  efficiency  losses  occur,  the  impacts  would  be  less 
under  the  policy  with  fewer  multi-skilled  personnel. 

Overall,  our  results  support  the  hypothesis  of  productivity  increases  and  cost  savings  from  multi¬ 
skilling  policies  proposed  in  the  Ogden  Air  Logistics  Complex’s  2012  BCA.  However,  we  found  that 
multi-skilling  all  five  of  the  non-painter  FWS  maintenance  specialties  into  the  painter  skill,  as  prescribed 
by  the  BCA,  is  not  a  desirable  policy  in  terms  of  flow  day  reductions,  annual  aircraft  throughput,  and  cost. 

5.2  Future  State  Results 

The  depot  plans  to  accommodate  their  future  state  workload  by  doubling  their  current  workforce  staffing. 
We  found  that  this  plan  would  let  them  achieve  their  projected  flow  day  goals,  but  at  high  labor  cost.  We 
therefore  created  a  “trade  space”  by  establishing  a  15%  smaller  workforce  as  baseline  for  our  experi¬ 
ments.  A  multi-skilling  experiment  was  then  conducted  on  the  reduced  workforce  baseline  and  personnel 
were  re-added  to  the  best  multi-skilled  scenarios.  The  goal  of  this  group  of  experiments  was  to  determine 
whether  targeted  multi-skilling  policies  allow  for  a  reduction  in  future  maintenance  personnel  require¬ 
ments.  The  future  state  results  summary  is  displayed  in  Table  2. 

Table  2:  Future  state  results  summary. 


Future  State  Experiments-  Results  Summary 

Experiment  Name  (Level) 

Flow  Days 

Delta  w/ 
Target  Flow 
Days 

Annual 

Throughput 

Multi-skilled 

Personnel 

Total 

Personnel 

Annual  Cost 

E9  Multi-skill  Personnel  Add  Best  (G7) 

121.07 

-4.5 

22.04 

257 

357 

$25,806,296 

E6  Multi-skill  Personnel  Add  Best  (F4) 

121.89 

-3.7 

22.08 

348 

348 

$25,552,422 

E9  Multi-skill  Personnel  Add  (G6) 

122.97 

-2.6 

21.86 

255 

354 

$25,586,017 

E6  Multi-skill  Personnel  Add  (F3) 

123.16 

-2.5 

21.82 

345 

345 

$25,332,143 

Baseline  Model  (No  Reduction) 

123.33 

-2.3 

21.81 

0 

396 

$27,449,037 

E9  Multi-skill  Personnel  Add  (G5) 

123.65 

-2.0 

21.74 

252 

351 

$25,365,737 

E9  Multi-skill  Personnel  Add  Best  (G7)  98%  Efficient 

124.47 

-1.1 

21.61 

257 

357 

$25,806,296 

E6  Multi-skill  Personnel  Add  Best  (F4)  98%  Efficient 

124.75 

-0.9 

21.57 

348 

348 

$25,552,422 

E9  Multi-skill  Personnel  Add  Best  (G7)  95%  Efficient 

128.31 

2.7 

21.02 

257 

357 

$25,806,296 

E6  Multi-skill  Personnel  Add  Best  (F4)  95%  Efficient 

128.46 

2.9 

20.92 

348 

348 

$25,552,422 

Personnel  Reduction  5%  (Dl) 

129.08 

3.5 

20.86 

0 

378 

$26,201,354 

Personnel  Reduction  10%  (D2) 

132.92 

7.3 

20.28 

0 

357 

$24,745,723 

E6  Multi-skill  Personnel  Add  Best  (F4)  90%  Efficient 

135.5 

9.9 

19.88 

348 

348 

$25,552,422 

E9  Multi-skill  Personnel  Add  Best  (G7)  90%  Efficient 

135.88 

10.3 

19.82 

257 

357 

$25,806,296 

Average  Flow  Day  Target  w/Overtime 

125.61 

The  results  show  that  multi-skilled  workforce  scenarios  with  no  efficiency  loss  and  between  39  and 
51  fewer  personnel  produce  results  superior  to  the  baseline  model  with  no  personnel  reductions. 
Additionally,  the  second  best  multi-skilled  experiment  produces  statistically  equivalent  flow  day  outputs 
to  the  baseline  with  a  reduction  in  workforce  of  51  personnel  and  a  cost  avoidance/savings  of  $2,1 17,000 
annually. 

Even  with  a  2  percent  increase  in  all  daily  maintenance  processing  times  reflected  by  a  less-efficient 
workforce,  the  best  multi-skilled  scenarios  beat  flow  day  goals  by  approximately  1  day.  A  5  percent  in¬ 
crease  in  all  processing  times  misses  the  flow  day  target  by  2.7  and  2.9  days.  However,  both  scenarios 
provide  a  cost  avoidance/savings  of  at  least  $1,643,000  per  year.  The  cost  difference  provides  manage¬ 
ment  flexibility  to  add  overtime  or  personnel  to  compensate  for  the  increased  processing  times.  The  ex¬ 
periments  and  analysis  of  the  current  and  future  state  models  provide  valuable  insights  into  the  benefits 
associated  with  multi-skilling  personnel  within  depot  operations. 
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6  CONCLUSION 

We  show  that  in  both  current  and  future  operational  states  for  F-22  depot  operations,  targeted  multi¬ 
skilling  policies  with  no  productivity  losses  increase  aircraft  throughput  to  levels  exceeding  the  current 
overtime  policies  with  significant  cost  savings/avoidance.  Furthermore,  our  research  shows  that  the  best 
performing  multi-skilling  policies  are  cost  favorable  for  a  given  level  of  aircraft  throughput  down  to  the 
95  percent  efficiency  level.  This  means  that  multi-skilling  is  favorable  up  to  the  point  that  all  required 
maintenance  times  grow  by  5  percent  due  to  productivity  losses  associated  with  a  more  generalized  work¬ 
force.  At  that  point,  overtime  and  multi-skilling  become  cost  equivalent  with  further  processing  time  in¬ 
creases  causing  overtime  to  be  more  desirable. 

Additionally,  employee  utilization  and  available  labor  hours  improve  significantly  with  multi-skilling, 
as  is  indicated  by  our  experimental  results  showing  significant  improvements  in  aircraft  flow  day  and 
throughput  measures  with  a  multi-skilled  workforce.  In  both  the  current  and  future  state  experiments,  a 
multi-skilled  workforce  provided  more  annual  throughput  and  direct  labor  hour  usage  than  a  workforce  of 
equal  or  greater  size  with  no  multi-skilling. 

A  limitation  of  our  study  is  that  the  multi-skilling  cost  calculations  did  not  include  time  considera¬ 
tions  to  obtain  security  clearances  or  additional  training.  Additionally,  the  aggregate  level  of  data  used 
drove  us  to  make  additional  assumptions  and  create  work-arounds.  Finally,  time  constraints  and  lengthy 
model  run  time  limited  our  analysis  to  marginal  analysis  with  associated  one  factor-at-a-time  adjustments, 
which  do  not  give  a  full  picture  of  the  possible  interactions  and  interdependencies  of  input  variables  and 
their  impact  on  outputs. 

We  conclude  by  identifying  opportunities  for  further  research.  Our  first  recommendation  is  to  model 
future  weapon  system  depot  processes  during  the  design  phase  to  assess  personnel  requirements  and  po¬ 
tential  multi-skilling  levels  and  benefits.  For  these  simulations,  we  recommend  using  software  with  more 
robust  capabilities  for  resource  related  scheduling  and  seizing  (possibly  an  agent-based  or  object  oriented 
simulation  tool).  Finally,  the  Air  Force  should  investigate  methods  for  implementing  a  flexible  (multi- 
skilled)  workforce  to  decrease  the  impact  of  proficiency  losses  associated  with  skill  generalization. 

DISCLAIMER 

The  views  expressed  in  this  paper  are  those  of  the  authors  and  do  not  reflect  the  official  policy  or  position 
of  the  United  States  Air  Force,  Department  of  Defense,  or  the  United  States  Government. 
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